Detecting sarcasm and verbal irony from people's subjective statements is crucial to understanding their intended meanings and real sentiments and positions in social scenarios. This paper describes the X-PuDu system that participated in SemEval-2022 Task 6, iSarcasmEval - Intended Sarcasm Detection in English and Arabic, which aims at detecting intended sarcasm in various settings of natural language understanding. Our solution finetunes pre-trained language models, such as ERNIE-M and DeBERTa, under the multilingual settings to recognize the irony from Arabic and English texts. Our system ranked second out of 43, and ninth out of 32 in Task A: one-sentence detection in English and Arabic; fifth out of 22 in Task B: binary multi-label classification in English; first out of 16, and fifth out of 13 in Task C: sentence-pair detection in English and Arabic.

在各种设备上部署深度学习模型已成为一个重要的话题。硬件专业化的浪潮为多维张量计算带来了一套多样化的加速度原始图。这些新的加速原始基原料以及新兴的机器学习模型带来了巨大的工程挑战。在本文中，我们提出了Tensorir，这是一种编译器抽象，用于通过这些张量计算原始素优化程序。Tensorir概括了现有机器学习编译器中使用的循环巢表示，以将张量计算作为一流的公民。最后，我们在抽象之上构建了一个端到端框架，以自动优化给定的张量计算原始图的深度学习模型。实验结果表明，Tensorir编译会自动使用给定硬件后端的张量计算原始图，并提供与跨平台的最新手工精制系统竞争性能的性能。

医疗自动诊断系统旨在模仿真实诊断过程中的人类医生。该任务被制定为具有症状探究和疾病诊断的顺序决策问题。近年来，许多研究人员使用了加强学习方法来处理这项任务。然而，最近的作品忽略了区分症状询问和疾病诊断行为并将它们混合到一个动作空间中。这导致对此任务的加固学习方法表现不满。此外，缺乏包含各种疾病和相应信息的公共评估数据集。为了解决这些问题，我们首先提出了一种新的医学自动诊断方法，分别与症状查询和疾病诊断分别作为加强学习任务和分类任务。我们还提出了一种强大而自适应的方法，可以使用分布熵作为媒体对齐两项任务。然后，我们创建从MedlinePlus知识库中提取的新数据集。数据集包含更多疾病和更完整的症状信息。模拟的实验患者更加现实。实验评价结果表明，我们的方法通过几个询问转弯实现了更高的医学诊断准确性，我们的方法在不同的数据集中优于三种最先进的方法。

对抗性的例子揭示了神经网络的脆弱性和不明原因的性质。研究对抗性实例的辩护具有相当大的实际重要性。大多数逆势的例子，错误分类网络通常无法被人类不可检测。在本文中，我们提出了一种防御模型，将分类器培训成具有形状偏好的人类感知分类模型。包括纹理传输网络（TTN）和辅助防御生成的对冲网络（GAN）的所提出的模型被称为人类感知辅助防御GaN（had-GaN）。 TTN用于扩展清洁图像的纹理样本，并有助于分类器聚焦在其形状上。 GaN用于为模型形成培训框架并生成必要的图像。在MNIST，时尚 - MNIST和CIFAR10上进行的一系列实验表明，所提出的模型优于网络鲁棒性的最先进的防御方法。该模型还证明了对抗性实例的防御能力的显着改善。

With increasing scale, large language models demonstrate both quantitative improvement and new qualitative capabilities, especially as zero-shot learners, like GPT-3. However, these results rely heavily on delicate prompt design and large computation. In this work, we explore whether the strong zero-shot ability could be achieved at a smaller model scale without any external supervised data. To achieve this goal, we revisit masked language modeling and present a geometry-guided self-supervised learning method (Go-tuningfor short) by taking a small number of task-aware self-supervised data to update language models further. Experiments show that Go-tuning can enable T5-small (80M) competitive zero-shot results compared with large language models, such as T5-XL (3B). We also apply Go-tuning on multi-task settings and develop a multi-task model, mgo-T5 (250M). It can reach the average performance of OPT (175B) on 9 datasets.

Diffusion model, a new generative modelling paradigm, has achieved great success in image, audio, and video generation. However, considering the discrete categorical nature of text, it is not trivial to extend continuous diffusion models to natural language, and text diffusion models are less studied. Sequence-to-sequence text generation is one of the essential natural language processing topics. In this work, we apply diffusion models to approach sequence-to-sequence text generation, and explore whether the superiority generation performance of diffusion model can transfer to natural language domain. We propose SeqDiffuSeq, a text diffusion model for sequence-to-sequence generation. SeqDiffuSeq uses an encoder-decoder Transformers architecture to model denoising function. In order to improve generation quality, SeqDiffuSeq combines the self-conditioning technique and a newly proposed adaptive noise schedule technique. The adaptive noise schedule has the difficulty of denoising evenly distributed across time steps, and considers exclusive noise schedules for tokens at different positional order. Experiment results illustrate the good performance on sequence-to-sequence generation in terms of text quality and inference time.

In this paper, we introduce a novel optimization algorithm for machine learning model training called Normalized Stochastic Gradient Descent (NSGD) inspired by Normalized Least Mean Squares (NLMS) from adaptive filtering. When we train a high-complexity model on a large dataset, the learning rate is significantly important as a poor choice of optimizer parameters can lead to divergence. The algorithm updates the new set of network weights using the stochastic gradient but with $\ell_1$ and $\ell_2$-based normalizations on the learning rate parameter similar to the NLMS algorithm. Our main difference from the existing normalization methods is that we do not include the error term in the normalization process. We normalize the update term using the input vector to the neuron. Our experiments present that the model can be trained to a better accuracy level on different initial settings using our optimization algorithm. In this paper, we demonstrate the efficiency of our training algorithm using ResNet-20 and a toy neural network on different benchmark datasets with different initializations. The NSGD improves the accuracy of the ResNet-20 from 91.96\% to 92.20\% on the CIFAR-10 dataset.

Language models with the Transformers structure have shown great performance in natural language processing. However, there still poses problems when fine-tuning pre-trained language models on downstream tasks, such as over-fitting or representation collapse. In this work, we propose HyPe, a simple yet effective fine-tuning technique to alleviate such problems by perturbing hidden representations of Transformers layers. Unlike previous works that only add noise to inputs or parameters, we argue that the hidden representations of Transformers layers convey more diverse and meaningful language information. Therefore, making the Transformers layers more robust to hidden representation perturbations can further benefit the fine-tuning of PLMs en bloc. We conduct extensive experiments and analyses on GLUE and other natural language inference datasets. Results demonstrate that HyPe outperforms vanilla fine-tuning and enhances generalization of hidden representations from different layers. In addition, HyPe acquires negligible computational overheads, and is better than and compatible with previous state-of-the-art fine-tuning techniques.

狗主人通常能够识别出揭示其狗的主观状态的行为线索，例如疼痛。但是自动识别疼痛状态非常具有挑战性。本文提出了一种基于视频的新型，两流深的神经网络方法，以解决此问题。我们提取和预处理身体关键点，并在视频中计算关键点和RGB表示的功能。我们提出了一种处理自我十分和缺少关键点的方法。我们还提出了一个由兽医专业人员收集的独特基于视频的狗行为数据集，并注释以进行疼痛，并通过建议的方法报告良好的分类结果。这项研究是基于机器学习的狗疼痛状态估计的第一批作品之一。

我们引入了基于高斯工艺回归和边缘化图内核（GPR-MGK）的探索性主动学习（AL）算法，以最低成本探索化学空间。使用高通量分子动力学模拟生成数据和图神经网络（GNN）以预测，我们为热力学性质预测构建了一个主动学习分子模拟框架。在特定的靶向251,728个烷烃分子中，由4至19个碳原子及其液体物理特性组成：密度，热能和汽化焓，我们使用AL算法选择最有用的分子来代表化学空间。计算和实验测试集的验证表明，只有313个（占总数的0.124 \％）分子足以训练用于计算测试集的$ \ rm r^2> 0.99 $的精确GNN模型和$ \ rm rm r^2>>实验测试集0.94 $。我们重点介绍了提出的AL算法的两个优点：与高通量数据生成和可靠的不确定性量化的兼容性。